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(57) Abstract: A security monitoring system including; an 
exit and entrance camera located at the exit and entrance of a 
structure; a detector for detecting the exit or entry of an indi- 
vidual; an image recording system for storing images from the 
exit and entrance cameras; and a computer vision system for 
analyzing the stored images using a predetermined criteria to 
determine if the exiling and entering individuals are the same. 
Preferably, the cameras are video cameras and the detector is 
cither a door contact sensor or an infra-red. The system prefer- 
ably transmits an alarm signal and/or at least a portion of the 
stored images to a remote site if the exiting and entering in- 
dividual is determined not to be the same. In a preferred im- 
plementation, the system also includes a database for storing 
face image data for each authorized individual of the struc- 
ture; and a face recognition system for comparing the stored 
images from the entry and entrance camera with the stored im- 
age data in the database and for determining if the exiling and 
entering individual is one of the authorized individuals. In this 
implementation, the alarm signal and/or at least a portion of 
the stored images is transmitted to a remote site if the exiting 
and entering individual is or is not determined to be the same 
and the exiting and entering individual is determined not to be 
one of the authorized individuals. 
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Apparatus and methods for resolution of entry/exit conflicts for security monitoring systems 



BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates generally to security monitoring systems and, 
more particularly, to a security monitoring system having a computer vision system for 
5 recognizing individuals entering and leaving a structure to resolve entry/exit conflicts. 

2. Prior Art 

Security monitoring systems of the prior art, particularly residential security 
systems, utilize a box that monitors contact sensors for doors and windows and one or more 

1 0 infra-red sensors for area monitoring. When a contact is triggered or an infra-red sensor 
triggers, an alarm is sounded and a signal is sent via a data link such as a phone line to a 
central monitoring site. The central monitoring site typically initiates a set of phone calls, to 
the homeowner, to work, and/or to a designated neighbor to determine if the alarm signal was 
due to an unauthorized intruder or just to an accidental triggering by a family member or 

1 5 other authorized occupant of the structure. 

If the alarm signal cannot be resolved by the phone calls, it is passed to the 
local police department. 95% of the calls passed to the police department turn out to be 
"false alarms" in the sense that they were not due to an unauthorized intruder. 

Significant portions of "false alarms" are caused by what is referred to as 

20 entry/exit conflicts. For instance, in the situation of a residential alarm system, the 

homeowner or other authorized occupant of a residence often arms the security system while 
leaving the residence and shortly thereafter realizes that they have forgotten something in the 
residence. As they return to the residence, they enter without disarming the system thereby 
causing an alarm to be sounded and/or an alarm signal to be sent to a central monitoring site. 

25 Similarly, the homeowner may arm the security system and remain inside the residence, such 
as during the night and may thereafter leave to get something outside the residence, i.e., the 
morning paper, thereby triggering a false alarm. 

In view of the prior art, there is a need for a security monitoring system, which 
resolves these and other types of entry/exit conflicts. 
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SUMMARY OF THE INVENTION 

Therefore it is an object of the present invention to provide a security 
monitoring system, which resolves the entry/exit conflicts of the prior art security monitoring 
5 systems. 

It is a further object of the present invention to provide a security monitoring 
system that monitors the entrance and exit of individuals to a structure, records key image 
data, and matches an exiting individual against an entering individual. 

Accordingly, a security monitoring system is provided. The security 

10 monitoring system comprises; an exit and entrance camera located at the exit and entrance, 
respectively of a structure; a detector for detecting the exit or entry of an individual into or 
from the structure; an image recording system for storing images from the exit and entrance 
cameras; and a computer vision system for analyzing the stored images transmitted from the 
image recording system using a predetermined criteria to determine if the exiting and 

1 5 entering individuals are the same. Preferably, the exit and entrance cameras are either video 
cameras or the images are video images or digital cameras and the images are still images. 

The security monitoring system preferably further comprises means for 
transmitting an alarm signal and/or at least a portion of the stored images to a remote site if 
the exiting and entering individual is detemiined not to be the same. Alternatively, the 

20 security monitoring system can transmit an alarm signal and/or at least a portion of the stored 
images if the exiting and entering individual is the same. 

In a preferred implementation of the security monitoring system of the present 
invention, the predetermined criteria used to determine if the exiting and entering individuals 
are the same comprises matching descriptive features of the entering and exiting individual 

25 such as a color, shape, or texture distribution of the stored images. 

In yet a more preferred implementation of the security monitoring system of 
the present invention, the security monitoring system further comprises: a database for 
storing image data for each authorized individual of the structure; and a recognition system 
for comparing the stored images from the entry and entrance camera with the stored image 

30 data in the database and for determining if the exiting and entering individual is one of the 
authorized individuals. Preferably, the stored image data in the database are face images and 
the recognition system is a face recognition system. In the more preferred implementation, 
the security monitoring system further comprises means for transmitting an alarm signal 
and/or at least a portion of the stored images to a remote site if the exiting and entering 
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individual is determined not to be the same and the exiting and entering individual is 
determined not to be one of the authorized individuals. 

Also provided is a method for performing security monitoring of a structure. 
The method comprising the steps of; capturing images at both an exit and entrance of the 
5 structure; before or after the capturing step, detecting the exit or entry of an individual into or 
from the structure; storing the images from the exit and entrance camera; and analyzing the 
stored images using a predetermined criteria to determine if the exiting and entering 
individuals are the same. 

Preferably, the method further comprises the step of transmitting an alarm 

10 signal and/or at least a portion of the stored images to a remote site if the exiting and entering 
individual is determined not to be the same. Alternatively, the security monitoring system 
can transmit an alarm signal and/or at least a portion of the stored images if the exiting and 
entering individual is the same. 

In a preferred implementation of the methods of the present invention, the 

1 5 method further comprises the steps of: storing image data for each authorized individual of 
the structure; and comparing the stored images from the entry and entrance with the stored 
image data for determining if the exiting and entering individual is one of the authorized 
individuals. The preferred implementation of the methods of the present invention further 
comprises the step of transmitting an alarm signal and/or at least a portion of the stored 

20 images to a remote site if the exiting and entering individual is determined not to be the same 
and the exiting and entering individual is determined not to be one of the authorized 
individuals. 



BRIEF DESCRIPTION OF THE DRAWINGS 
25 These and other features, aspects, and advantages of the apparatus and 

methods of the present invention will become better understood with regard to the following 
description, appended claims, and accompanying drawings where: 

Figure 1 illustrates a schematical view of the security monitoring system of the 
present invention. 

30 Figure 2 illustrates a flow chart of a first entry/exit conflict to be resolved by 

the security monitoring system of Figure 1. 

Figure 3 illustrates a flow chart of a second entry/exit conflict to be resolved 
by the security monitoring system of Figure 1. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

Although this invention is applicable to numerous and various types of 
security monitoring systems, it has been found particularly useful in the environment of 
residential security monitoring systems. Therefore, without limiting the applicability of the 
5 invention to residential security monitoring systems, the invention will be described in such 
environment. 

Referring now to Figure 1, there is illustrated a security monitoring system of 
the present invention, generally referred to by reference numeral 100. The security 
monitoring system 100 includes an exit and entrance camera 102, 104 located at the exit 102a 

10 and entrance 104a of a door 106, respectively, of a residential home or a commercial 

building, generally referred to herein as a structure 108. The door 106 is shown as a typical 
exterior door by way of example only and not to limit the scope of the invention. Those of 
skill in the art will recognize that any thing providing an entry or exit into or from a structure 
can be the subject of the present invention, such as a sliding door, a roll-up door, or a garage 

15 door. The exit and entrance cameras 102, 1 04 are preferably video cameras which supply a 
sequence of video frames, but can alternatively be digital cameras supplying a sequence of 
still pictures. Furthermore, the exit.and entrance cameras 102, 104 can be static or pan-tilt- 
zoom (PTZ) types. Both the sequence of video frame and still pictures are generally referred 
to herein as images or image data. The exit and entrance cameras 102, 104 are shown 

20 mounted near the door 106 and facing towards the exit and entrance 102a, 104a, respectively. 
However, this configuration is illustrated for its simplicity. Preferably, the exit and entrance 
cameras 102, 104 are mounted in the door 106 as is described in co-pending U.S. Patent 
Application No. 09/734780 (Docket No. 701662, 1 3935) which is incorporated herein by its 
reference. 

25 The security monitoring system also includes an alarm system 110. The alarm 

system 1 10 has a detector for detecting the exit or entry of an individual through the door 106 
and into or from the structure 108. The detector can be any device known in the art for 
detecting the operation of a door 106, such as a magnetic door contact sensor 1 12 positioned 
on the door 106 or an infra-red sensor 1 14 located near the door 106 of the structure. 

30 Alternatively, the detector can be a sensor located on the door-knob of the door 1 06 or a 
background change detection system such as is also disclosed in copending U.S. Patent 
Application Serial Number 09/734780 (Attorney Docket 701662, 13935). The detector can 
also be the act of arming the alarm system, or turning the door-knob or key tumbler on the 
door 106 or any device which can serve as a detector. 
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An image recording system 116 stores the images from the exit and entrance 
cameras upon the triggering of the detector. That is, if the detector is triggered such that an 
alarm signal would ordinarily be transmitted to a central monitoring site 1 18, the image 
recording system 116 stores the images from the exit and entrance cameras 102, 104. 
5 Alternatively, the image recording system 1 16 can store all of the images in a buffer where 
new image data overwrites old image data such that a predetermined amount of current image 
data is always stored and available. It is preferred that the image recording system 1 16 is a 
computer or any other processor having a storage device such as a hard drive and an image 
capture card. However, those skilled in the art will recognize that the image recording 

1 0 system 1 1 6 can be of any type known in the art without departing from the scope and spirit of 
the present invention. 

A computer vision system 120 analyzes the stored images transmitted from the 
image recording system 116 using a predetermined criteria to determine if the exiting and 
entering individuals are the same. Preferably, the computer vision algorithm extracts 

1 5 descriptive features of the individual. These descriptive features are preferably based on one 
or a combination of the color, texture, or shape distribution of the images. Preferably, the 
descriptive features are focused on a particular area of the individual, such as his or her 
clothes. The descriptive features of the images of the individuals entering and leaving are 
extracted and compared. If there is no match between the entering and leaving individuals, 

20 an alarm signal is triggered and transmitted to the central monitoring site 1 1 8 via a data link 
119, such as a telephone line (POTS). Preferably the image data of the entering or leaving 
individual is also transmitted to the central monitoring site 118. It is preferred that computer 
vision system 120 is a system such as that disclosed in copending U.S. Application Serial 
Number 09/703423 (Attorney Docket 701254), which is incorporated herein by its reference. 

25 However, those skilled in the art will recognize that the computer vision system 120 can be 
of any type known in the art without departing from the scope and spirit of the present 
invention, such as that disclosed in Raja et al, "Segmentation and Tracking Using Colour 
Mixture Models", 3 rd Asian Conference on Computer Vision, Vol. 1, pp. 607-614, Hong 
Kong, China, January 1998, which is also incorporated herein by its reference. 

30 In a preferred implementation of the security monitoring system 1 00 of the 

present invention, there is also included a database 122 for storing image data, such as the 
face, for each authorized individual of the structure. In the case of a residential home, the 
faces of the people living in the home are recorded, preferably by the exit and entry cameras 
102, 104 and stored in the database 122. Where the computer vision system 120 determines 
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that the exiting and entering individuals are not the same, instead of triggering an alarm, the 
security monitoring system 100 can alternatively utilize a recognition system 124 which 
compares the stored images from the entry and entrance camera with the stored image data 
(e.g., faces) in the database 122. If there is no match between the stored images from the 
5 entry and entrance camera with the stored image data in the database 1 22 then the system 
proceeds as described above by triggering an alarm signal and/or transmitting image data of 
the entering or leaving individual to the central monitoring site 118. Face recognition 
algorithms and systems are well known in the art, such as the one described by S. Gutta et al., 
Face Recognition, Sixth International Conference on Computer Vision, pgs. 646-651, IEEE, 

1 0 January 4-7, 1 998, Mumbai, India. 

If either the comparison of the entering and leaving individuals from the 
computer vision system 120 or the comparison of the stored images from the entry and 
entrance camera with the stored image data in the database 122 results in a match, the alarm 
signal is not triggered (i.e., an exit or entry that would ordinarily trigger an alarm is ignored). 

1 5 Alternatively, the alarm signal can be transmitted upon the "unauthorized" exit or entry and 
an ignore signal can follow if it is determined that the entry and exit individuals are the same. 
Furthermore, even if the exiting and entering individuals are determined to be the same, the 
alarm signal and/or a portion of the image data can be transmitted to the central monitoring 
site. For instance, if the individual is determined to be the maid, who is authorized to be in 

20 the structure, the image data can later support or deny a subsequent allegation that the maid 
stole property from the structure. 

A combination of face as well as descriptive features of the individuals can 
also be used as a way to verify the individual as being authorized. This can be done either 
sequentially as discussed below or in parallel. 

25 The security monitoring system 1 00 has been described with reference to 

Figure 1 in general terms without reference to a particular entry/exit conflict. Two particular 
exit/entry conflicts will now be described with reference to Figures 2 and 3. However, those 
skilled in the art will realize that the general description above is intended to cover any such 
exit/entry conflict and is not therefore limited to any one particular entry/exit conflict or to 

30 the entry/exit conflicts described with regard to Figures 2 and 3 . 

Referring now to the flowchart of Figure 2, there is illustrated a method for 
providing the security monitoring system of the present invention where the alarm system 
1 10 is "armed" and an entrance into the structure 108 will cause the triggering of the alarm 
and transmission of an alarm signal to the central monitoring site 118. Such is the case where 
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an authorized person "arms" the alarm system 1 10, exits the structure 108, and returns a short 
time later without disarming the alarm system 1 10 because they have forgotten something 
inside the structure 108. 

At step 200, the exit and entrance cameras 102, 104 capture images from the 

5 exit and entrance 1 02a, 1 04a of the door 1 06 of the structure 108. Alternatively, the cameras 
can come on to capture image data upon the occurrence of an event, such as the turning of a 
door knob or key tumbler on the door 1 06 or triggering a sensor on or near the door 106 or 
detecting through background models. At step 202 one or more of the detectors 1 12, 1 14 
check for the detection of an exiting individual. If there is a detection of an exit (shown 

1 0 schematically as path 204b), the exit camera 1 02 will store a last predetermined portion of the 
video frames or still images from a buffer (which may also be database 122) at step 204. 
Alternatively, the exit camera 102 can capture images and the image recording system 1 16 
store those images upon the detection of an event, such as when the authorized person "arms" 
the alarm system in which case the exit camera 102 can capture and record the arming of the 

1 5 alarm system. If there is no detection of an exit, the flow chart proceeds along path 204a and 
the cameras 102, 104 continue to capture images or alternatively return to a dormant state 
before capturing images upon the occurrence of the event. 

The security monitoring system 100 then looks for a re-entry into the structure 
at step 208. If the alarm system 1 10 detects a re-entry into the structure within a 

20 predetermined time limit (shown schematically as path 208b), the images from the entry 
camera 104 are stored or taken from the buffer into the image recording device 1 16 at step 
210. If there is no re-entry, the system continues along path 208a to step 200 to capture 
images from the exit and entrance cameras. 

The stored images from the exit and entry cameras 102, 104 are then analyzed 

25 by the computer vision system 120 at step 212 to determine if the descriptive features therein 
match at step 214. If there is a match (i.e., it is determined that the same person who left, 
later re-entered) the system continues along path 214a and the detection of the re-entry is 
ignored at step 218. If there is not a match, the system can follow along either path 214c or 
214b. If the system follows along path 214c (shown as a dotted line) to step 216, an alarm 

30 signal is transmitted preferably along with the stored images from the entrance camera to the 
central monitoring site 118. However, in case the exiting person re-enters with a different 
descriptive feature, as is the case where a color distribution is used and the person exits with 
a jacket and re-enters without it, a further face recognition can be performed at step 220. At 
step 220, the stored image data, such as the faces of all of the people authorized to have 
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access to the structure, are compared to the stored images from the entrance camera 104. At 
step 222, it is determined if there is a match. If there is no match, the system proceeds along 
path 222b to step 216 where an alarm is triggered. If there is a match, the system proceeds 
along path 222a to step 218 where the detection of the re-entry is ignored or alternatively, 

5 along path 222c (shown as a dotted line) where the alarm signal and/or image data can also 
be transmitted to the central monitoring site as discussed above. 

Referring now to the flowchart of Figure 3, there is illustrated a method for 
providing the security monitoring system of the present invention where the alarm system 
1 10 is "armed" and an exit from the structure 1 08 will cause the triggering of the alarm and 

10 transmission of an alarm signal to the central monitoring site 118. Such is the case where an 
authorized person "arms" the alarm system during the night, exits the structure upon waking 
without disarming the alarm system, and returns a short time later, perhaps with the morning 
paper. Similar reference numerals in Figure 3 denote similar steps as described above with 
reference to Figure 2. 

1 5 At step 200, the exit and entry cameras 1 02, 1 04 capture images from the exit 

and entrance 102a, 104a of the door 106 of the structure 108. At step 202 one or more of the 
detectors 1 12, 1 14 check for the detection of an exiting individual. If there is a detection of 
an exit (shown schematically as path 204b), the exit camera 102 stores the video frames or 
still images at step 204. If there is no detection of an exit, the flow chart proceeds along path 

20 204a and continues to capture images or the cameras 102, 1 04 return to a dormant state until 
the occurrence of an event. 

The security monitoring system 1 00 then looks for a re-entry into the structure 
at step 208. If the alarm system 110 detects a re-entry into the structure within a 
predetermined time limit (shown schematically as path 208b), the last portion of images from 

25 the entry camera 104 are stored from a buffer into the image recording device 1 16 at step 

210. If there is no re-entry, the system continues along path 208c to step 216 where an alarm 
signal is triggered. 

The stored images from the exit and entry cameras 102, 104 are then analyzed 
by the computer vision system 120 at step 212 to determine if the descriptive features therein 
30 match at step 214. If there is a match (i.e., it is determined that the same person who left, 

later re-entered) the system continues along path 214a and the detection of the exit is ignored 
at step 300. If there is not a match, the system can follow along either path 214c or 214b. If 
the system follows along path 214c (shown as a dotted line) to step 216, an alarm signal is 
transmitted preferably along with the stored images from the exit camera to the central 
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monitoring site 118. However, in case the exiting person re-enters with a different 
descriptive feature, as is the case where a color distribution is used and the person exits with 
a jacket and re-enters without it, a further face recognition can be performed at step 220. At 
step 220, the stored image data, such as the faces of all of the people authorized to have 
5 access to the structure, are compared to the stored images from the entrance camera 104. At 
step 222, it is determined if there is a match. If there is no match the system proceeds along 
path 222b to step 216 where an alarm is triggered. If there is a match, the system proceeds 
along path 222a to step 300 where the detection of the exit is ignored or alternatively, along 
path 222c (shown as a dotted line) where the alarm signal and/or image data can also be 

1 0 transmitted to the central monitoring site as discussed above. 

Those skilled in the art will recognize that the system illustrated in Figures 2 
and 3 resolve the entry/exit conflicts described above. That is, a false alarm will not be 
triggered where an authorized person leaves and arms the alarm system but re-enters without 
disarming the alarm system or leaves while the alarm system is armed and returns. 

1 5 While there has been shown and described what is considered to be preferred 

embodiments of the invention, it will, of course, be understood that various modifications and 
changes in form or detail could readily be made without departing from the spirit of the 
invention. It is therefore intended that the invention be not limited to the exact forms 
described and illustrated, but should be constructed to cover all modifications that may fall 

20 within the scope of the appended claims. 
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1 . A security monitoring system (1 00) comprising; 

an exit and entrance camera (102, 104) located at the exit (102a) and entrance 
(104a), respectively of a structure (108); 

a detector (1 12, 1 14) for detecting the exit or entry of an individual into or 
5 from the structure (108); 

an image recording system (116) for storing images from the exit and entrance 
cameras (102, 104); and 

a computer vision system (120) for analyzing the stored images transmitted 
from the image recording system (116) using a predetermined criteria to determine if the 
1 0 exiting and entering individuals are the same. 

2. The security monitoring system of claim 1 , wherein the detector is a door 
contact sensor (1 12) located on an exit/entry door (106) of the structure (108). 



3 . The security monitoring system of claim 1 , wherein the detector is an infra-red 
sensor (1 14) located near an exit/entry door (106) of the structure (108). 

4. The security monitoring system of claim 1 , further comprising means for 
transmitting an alarm signal if the exiting and entering individual is determined not to be the 



5. The security monitoring system of claim 1, further comprising means for 

transmitting at least a portion of the stored images to a remote site (118) if the exiting and 
entering individual is determined not to be the same. 



6. The security monitoring system of claim 1 , wherein the predetermined criteria 

used to determine if the exiting and entering individuals are the same comprises matching 
descriptive features of the entering and exiting individual. 
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7. The security monitoring system of claim 6, wherein the descriptive feature is 

selected from a group consisting of one or more of a color, shape, or texture distribution of 
the stored images. 

5 8. The security monitoring system of claim 1 , further comprising: 

a database (122) for storing image data for each authorized individual of the 
structure (108); and 

a recognition system (124) for comparing the stored images from the entry and 
entrance camera (102, 104) with the stored image data in the database (122) and for 
10 determining if the exiting and entering individual is one of the authorized individuals. 

9. The security monitoring system of claim 8, wherein the stored image data in 
the database (122) are face images and the recognition system (124) is a face recognition 
system. 

15 

1 0. The security monitoring system of claim 8, further comprising means for 
transmitting an alarm signal if the exiting and entering individual is determined not to be the 
same and the exiting and entering individual is determined not to be one of the authorized 
individuals. 

20 

1 1 . The security monitoring system of claim 8, further comprising means for 
transmitting at least a portion of the stored images to a remote site if the exiting and entering 
individual is determined not to be the same and the exiting and entering individual is 
determined not to be one of the authorized individuals. 

25 

12. The security monitoring system of claim 1 , wherein the image recording 
system (116) stores the images from the exit and entrance cameras (102, 104) upon the 
occurrence of an event. 

30 13. The security monitoring system of claim 12, wherein the occurrence of an 

event is the triggering of the detector (1 12, 1 14). 



14. A method for performing security monitoring of a structure (1 08), the method 

comprising the steps of; 
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capturing images at both an exit and entrance (102a, 104a) of the structure 

(108); 

before or after the capturing step, detecting the exit or entry of an individual 
into or from the structure (108); 
5 storing the captured images; and 

analyzing the stored images using a predetermined criteria to determine if the 
exiting and entering individuals are the same. 

15. The method of claim 14, further comprising the step of transmitting an alarm 
10 signal if the exiting and entering individual is determined not to be the same. 

1 6. The method of claim 1 4, further comprising the step of transmitting at least a 
portion of the stored images to a remote site (1 18) if the exiting and entering individual is 
determined not to be the same. 

15 

17. The method of claim 14, further comprising the steps of: 

storing image data for each authorized individual of the structure; and 
comparing the stored images from the entry and entrance (102a, 104a) with the 
stored image data for determining if the exiting and entering individual is one of the 
20 authorized individuals 



18. The method of claim 1 7, further comprising the step of transmitting an alarm 
signal if the exiting and entering individual is determined not to be the same and the exiting 
and entering individual is determined not to be one of the authorized individuals. 

25 

19. The method of claim 1 7, further comprising the step of transmitting at least a 
portion of the stored images to a remote site (118) if the exiting and entering individual is 
determined not to be the same and the exiting and entering individual is determined not to be 
one of the authorized individuals. 



20. The method of claim 14, wherein the storing step stores the images from exit 

and entrance cameras (102, 104) upon the occurrence of an event. 
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